The impact of 3-option responses to multiple-choice questions on guessing strategies and cut score determinations

Introduction: Research has asserted MCQ items using threeresponse options (one correct answer with two distractors) iscomparable to, and possibly preferable over, traditional MCQitem formats consisting of four response options (e.g., one correctanswer with three distractors), or five response options (e.g., onecorrect answer with four distractors). Some medical educatorshave also adopted the practice of using 3-option responseson MCQ exams as a response to the difficulty experienced ingenerating additional plausible distractors. To date, however,little work has explored how 3-option responses might impactvalidity threats stemming from random guessing strategies,and what impact 3-option responses might have on cut-scoredeterminations, particularly in the context of medical educationclassroom assessments. The purpose of this work is to furtherexplore these critically important considerations that largely havegone ignored in the medical education literature to this point.Methods: A cumulative binomial distribution formula was usedto calculate the probability that an examinee will answer atrandom a given number of items correctly on any exam (of anylength). By way of a demonstration, a variety of scenarios werepresented to illustrate how examination length and the number ofresponse options impact examinees’ chances of passing a givenexamination, and how subsequent cut-score decisions may beimpacted by these factors.Results: As a general rule, classroom assessments containingfewer items should utilize traditional 4-option or 5-optionresponses, whereas assessments of greater length are affordedgreater flexibility in potentially utilizing 3-option responses.Conclusions: More research on items with 3-option responses isneeded to better understand what value, if any, 3-option responsestruly add to classroom assessments, and in what contexts potentialbenefits might be discernible.Keywords: Medical education; Assessment; Evaluation; Psychometrics